feat: store logs in different folders based on the chain #3948

alessandromazza98 · 2023-07-27T09:53:29Z

When --log.persistent flag is enabled, logs are now stored in different folders based on the chain or in the already-chain-specific --datadir.

Examples (no --datadir flag):
~/.cache/reth/logs/mainnet/reth.log
~/.cache/reth/logs/sepolia/reth.log

Examples (with --datadir flag):
~/custom-data-dir/reth.log

codecov · 2023-07-27T22:08:56Z

Codecov Report

Merging #3948 (86deeff) into main (56b2ef1) will decrease coverage by 0.04%.
Report is 1 commits behind head on main.
The diff coverage is 65.00%.

Files Changed	Coverage Δ
bin/reth/src/dirs.rs	`71.01% <0.00%> (-1.58%)`	⬇️
bin/reth/src/cli.rs	`34.40% <76.47%> (+12.33%)`	⬆️

... and 8 files with indirect coverage changes

Flag	Coverage Δ
integration-tests	`16.34% <0.00%> (-0.01%)`	⬇️
unit-tests	`64.31% <65.00%> (-0.04%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
reth binary	`26.88% <65.00%> (+0.20%)`	⬆️
blockchain tree	`83.04% <ø> (ø)`
pipeline	`89.82% <ø> (ø)`
storage (db)	`74.30% <ø> (ø)`
trie	`94.70% <ø> (ø)`
txpool	`45.40% <ø> (-0.61%)`	⬇️
networking	`77.62% <ø> (-0.05%)`	⬇️
rpc	`58.52% <ø> (ø)`
consensus	`63.51% <ø> (ø)`
revm	`33.08% <ø> (ø)`
payload builder	`6.58% <ø> (ø)`
primitives	`87.92% <ø> (-0.02%)`	⬇️

Rjected

looks like a good start! is it possible to get a CLI test for this?

for example like the ones here:

reth/bin/reth/src/node/mod.rs

Lines 879 to 890 in f3a7ae1

    
           #[test] 
        
           fn parse_db_path() { 
        
               let cmd = Command::try_parse_from(["reth"]).unwrap(); 
        
               let data_dir = cmd.datadir.unwrap_or_chain_default(cmd.chain.chain); 
        
               let db_path = data_dir.db_path(); 
        
               assert!(db_path.ends_with("reth/mainnet/db"), "{:?}", cmd.config); 
        
               let cmd = Command::try_parse_from(["reth", "--datadir", "my/custom/path"]).unwrap(); 
        
               let data_dir = cmd.datadir.unwrap_or_chain_default(cmd.chain.chain); 
        
               let db_path = data_dir.db_path(); 
        
               assert_eq!(db_path, Path::new("my/custom/path/db")); 
        
           }

bin/reth/src/cli.rs

alessandromazza98 · 2023-07-30T16:53:23Z

At the end I chose the following option:

keep logs and datadir directories separate, always using the Logs directory for logs.

This actually turned out the best decision because you can delete some fields to the Cli struct: the datadir field because it is no longer needed since logs are always stored in their specific directory, which is tied to the chain's name.

I also added a test to verify that the directory is properly parsed.

Let me know what do you think.

onbjerg

Makes sense to me, thank you. There are some conflicts to solve

Rjected

Makes sense and looks good to me! thank you!

alessandromazza98 added 2 commits July 27, 2023 10:35

add network name to logs dir

c7cd890

move logs to the already-chain-specific --datadir

cc1aa58

alessandromazza98 requested a review from onbjerg as a code owner July 27, 2023 09:53

Rjected self-requested a review July 27, 2023 21:59

Rjected requested changes Jul 27, 2023

View reviewed changes

bin/reth/src/cli.rs Outdated Show resolved Hide resolved

bin/reth/src/cli.rs Outdated Show resolved Hide resolved

always use logs directory, add test

7ccba59

resolve conflicts with main

27e84c0

alessandromazza98 force-pushed the issue3858 branch from 1182d39 to 27e84c0 Compare July 30, 2023 17:13

onbjerg changed the title ~~store logs in different folders based on the chain~~ feat: store logs in different folders based on the chain Jul 31, 2023

onbjerg added C-enhancement New feature or request M-changelog This change should be included in the changelog A-observability Related to tracing, metrics, logs and other observability tools A-cli Related to the reth CLI labels Jul 31, 2023

onbjerg approved these changes Jul 31, 2023

View reviewed changes

Merge branch 'main' into issue3858

e56e322

onbjerg enabled auto-merge July 31, 2023 13:01

chore: fmt

86deeff

onbjerg requested a review from Rjected July 31, 2023 13:18

Rjected approved these changes Jul 31, 2023

View reviewed changes

onbjerg added this pull request to the merge queue Jul 31, 2023

Merged via the queue into paradigmxyz:main with commit 5823255 Jul 31, 2023

alessandromazza98 deleted the issue3858 branch August 1, 2023 08:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: store logs in different folders based on the chain #3948

feat: store logs in different folders based on the chain #3948

alessandromazza98 commented Jul 27, 2023

codecov bot commented Jul 27, 2023 •

edited

Loading

Rjected left a comment

alessandromazza98 commented Jul 30, 2023

onbjerg left a comment •

edited

Loading

Rjected left a comment •

edited

Loading

	#[test]
	fn parse_db_path() {
	let cmd = Command::try_parse_from(["reth"]).unwrap();
	let data_dir = cmd.datadir.unwrap_or_chain_default(cmd.chain.chain);
	let db_path = data_dir.db_path();
	assert!(db_path.ends_with("reth/mainnet/db"), "{:?}", cmd.config);

	let cmd = Command::try_parse_from(["reth", "--datadir", "my/custom/path"]).unwrap();
	let data_dir = cmd.datadir.unwrap_or_chain_default(cmd.chain.chain);
	let db_path = data_dir.db_path();
	assert_eq!(db_path, Path::new("my/custom/path/db"));
	}

feat: store logs in different folders based on the chain #3948

feat: store logs in different folders based on the chain #3948

Conversation

alessandromazza98 commented Jul 27, 2023

codecov bot commented Jul 27, 2023 • edited Loading

Codecov Report

Rjected left a comment

Choose a reason for hiding this comment

alessandromazza98 commented Jul 30, 2023

onbjerg left a comment • edited Loading

Choose a reason for hiding this comment

Rjected left a comment • edited Loading

Choose a reason for hiding this comment

codecov bot commented Jul 27, 2023 •

edited

Loading

onbjerg left a comment •

edited

Loading

Rjected left a comment •

edited

Loading